Skip to main content

Work with Typesense

All workflows and schemas related to typesense are maintained in visn-devops-scripts repository.

Steps to migrate index from algolia to typesense​

  1. Create a schema file for that index under src\schemas\typesense
  2. Add the fields that will be used for filtering, faceting, sorting etc. in the schema.(You can refer one of the existing schema)
  3. Add the newly created schema in src\schemas\typesenseSchemaLookUp.ts
  4. Update the .github\workflows\migrate-from-algolia-to-typesense.yaml workflow for all env. (Just need to update the index names)
  5. Run the workflow with a appropriate suffix (preferably release number) as input.
  1. Update the .github\workflows\DEV-backup-typesense-data.yaml backup workflow with new index. (DEV and PROD)

How to update or add a field in existing schema​

  1. First update or add the filed in the schema in code
  2. Use the Create collection workflow to create a new collection with updated schema
  3. Now use the Migrate to new collection workflow to migrate the data from old collection to new collection
  4. After all data is migrated successfully, use the update alias workflow to update the alias to point to new collection
  5. After successfully updating the alias, use the delete collection workflow to delete the old collection

Creating new collection​

Workflow Used: .github\workflows\DEV-create-typesense-collection.yaml

  1. Create a schema file for that index under src\schemas\typesense ( More about how to create schema can be found in references )
  2. Add the newly created schema in src\schemas\typesenseSchemaLookUp.ts (Keep the typesenseCollectionsConstant same as alias name)
  3. Next need to trigger the above mentioned workflow, with inputs as Collection name: Should be same as alias that is used for schema Suffix: Any name for example date (aug_2023) or release_11_0_0(release is recommended for higher environments)
  1. After successfully workflow run a collection will be created. Collection will follow collectionName_suffix format Eg. Collection Name: enquirySearchIndex Suffix: release_11_0_0 Final collection name: enquirySearchIndex_release_11_0_0

Note: if creating a new collection from existing schema then Step 1 & 2 can be skipped

Migrating data to new collection​

Workflow used: .github\workflows\DEV-migrate-to-new-collection.yaml

This workflow is used when we have updated a schema and created a new collection with updated schema, and want to move the data form the old collection to the newly created collection.

Note: Recommended to create a backup before running this workflow.

  1. Need to run the above mentioned workflow with following inputs. Alias: Alias name that is used for old collection (Same as collection name that was used during creating a collection) Suffix: suffix that was used when creating the new collection
  1. After this workflow runs successfully, need to verify that there are no failed documents to import( Refer the below screenshot)
  1. If there are failed documents need to fix those errors and re run the workflow.

Creating or updating an alias​

Workflow used: .github\workflows\DEV-update-typesense-alias.yaml

This workflow will be used to create or update an alias to point to new collection. You can read more about alias here.

  1. Use the above mentioned workflow with following inputs: collection: Full collection name, will be in the format alias_suffix (E.g. enquirySearchIndex_release_11_0_0) alias: The alias name the given collection should point to.
  1. After successfully running this workflow provided alias will be modified (or created if does not exits) pointing to the provided collection.

Deleting collection​

Workflow used: .github\workflows\DEV-delete-typesense-collection.yaml

This workflow will be used to delete a collection.

  1. Use the above mentioned workflow with following inputs: Collection: enquirySearchIndex_release_11_0_0 (full collection name should be provided)
  2. After successfully running this workflow the provided collection will be deleted

Note: if only the alias name is provided name as input, then the collection which the alias is pointing to will be deleted.

Backup​

Workflow used: .github\workflows\DEV-backup-typesense-data.yaml At the end of each day a scheduled workflow is run to create a backup of typesense data (Only in DEV and PROD environment).

References​

Status: Accepted
Category: Protected
Authored By: Gladson on Sep 25, 2023
Revisions.